-
-
Notifications
You must be signed in to change notification settings - Fork 329
chore: switch to monorepo, move realtime inside #1190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
I've decided to stick with `just` for command management, as they provide a module structure that's very favorable for monorepos. each package is going to be stored at `src/{package}`, with its own `pyproject.toml` and `mod.just` files, for defining relevant commands, tests, lints, and so on. the main `justfile` in the workspace root then serves as the coordinator, calling all the subfiles in order to create commands to run in the CI and pre-commit.
make `supabase-py` a sub package, and make the workspace root a virtual workspace that contains no "main package". this is in order to have more homogeneity around the packages, as before the `supabase-py` was treated specially (pyproject in the root instead of in `src/supabase`).
Great work and explanations here @o-santi. I really like the direction this is going but I think we need to ensure #1166 happens before this PR get's merged in. We would need to make at least two more releases with the current setup first before migration to this setup. Otherwise we will still be releasing two versions of the |
Of course, I just wanted to get some comments and suggestions before moving forward with this. In talking with @olirice, we came to the conclusion that it might make sense to leave the command runner part out of the PR for now, and after all the sub packages are merged in, we can try to find the best way to do it. |
Pull Request Test Coverage Report for Build 16940372634Details
💛 - Coveralls |
Now that both packages have been deprecated, we can finally move on with this MR. I believe for this to get merged there are two remaining questions:
@grdsdev @silentworks looping both of you in on this. What do you think? |
|
|
I vote for synchronizing versions with supabase-py independent of current version, and make it stand out in CHANGELOG that although it is a major bump, there aren't significant changes, and the bump was just for aligning versions with main library. |
I agree with @grdsdev here! |
unasync-cli uses setuptools.find_packages() under the hood to find the files to patch, which is not ideal given it does not know how to handle the `src` directories correctly. instead, by using `unasync` directly, we can give it the list of files to operate on
I agree with the version sync too. I think the packages that are pre 1.x are already stable and have feature parity with the JS libraries. |
What kind of change does this PR introduce?
This PR contains the first change in the monorepo switch, by moving
realtime
into this repository, and deprecating https://github.com/supabase/realtime-py (which I intend to do later). This requires restructuring this repository's files in order to leverageuv
workspaces, as explained in #1186.The changes can be summarized by the following tree diagram:
Reasoning
File structuring
uv
broadly expects python "packages" to be either libraries or applications; and for workspaces, it expects them to be structured as N libraries, whose directory need to be specified inuv.workspace.members
, and 1 application, located in./src/{appname}
. This means that projects in the root of the repository are generally interpreted as describing the application, but given that this repository shouldn't have no application, I decided to move most of the rootspyproject.toml
intosrc/supabase
, and use it as a workspace library entry by listing it inmembers
.uv
generally calls these "virtual workspaces" but I couldn't find much documentation around them anywhere.Package specific dependencies and commands
uv
already handles package specific dependencies by leveraginguv --exact --package {package} {cmd} ...
, so no additional care is required besides calling the command correctly. In order to make running these more streamlined, and to share/compose them into bigger ones (for instance, to run the same commands locally and in CI), I think it makes sense to rely on a command runner. I outlined the following broad requirements in order to decide which tool to use:uv
, including test/dev time dependencies (by usingdependency-groups
). This is to ensure that we are not specifying tests differently to run locally vs in CI.realtime
requires certain docker instances to be up before runningpytest
, whilesupabase
does not. On the flip side,supabase
relies onunasync-cli
to generate the_sync
, with adhocsed
commands, whilerealtime
does not use any of it - I don't think we want to keep this behavior in the future, but I do not intend to change it now.I decided to give just a try. It can be summarized as a
make
successor, that aims to be a command runner instead of a build system. By being a command runner, it does not suffer from issues thatMakefile
s suffer, like that targets may be interpreted as files, and thus need to be marked as PHONY sometimes, and has niceties such as aliases, readable errors, recipe arguments, invoking from anywhere (not only from the workspace root).just
also allows for structuring the files as namespaced modules, which helps to avoid hackier solutions to name collisions, like prepending every command with the package name. With this, each package can declare its commands for testing, linting and publishing insidesrc/{packge}/mod.just
, that is then imported as a module inside thejustfile
in the root of the workspace. This allows us to define targets likejust test
by composing each of the packages'test
, without any additional coordination.Alternatives
I believe that trying to use
Makefile
s to the same effect would result in a big spaghetti mess of commands, which is why I sought for a different tool.just
was the best tool I found that was also minimalistic and not an entire build system, but I'd love some more suggestions for monorepo related tooling, specially those that generally fit those guidelines. For the record, I considered the following tools:uv
to run it in the first place.uv
.